Service provider networks offer network-based services to fulfill users' computing-service needs without the users having to invest in and maintain computing infrastructure required to implement the services. These service provider networks may provide network-based computing resources on an as-needed basis. For example, a service provider network may permit users to purchase and utilize computing resources such as virtual machine (“VM”) instances, data storage resources, database resources, networking resources, network services, and other types of computing resources. Users may configure the computing resources provided by a service provider network to implement desired functionality, such as to provide a network-based application or another type of functionality.
Managing scalable computing resources provided by a service provider network such as those described above can be complex. For example, in order to service the variable amounts of requests received from users to execute commands within VM instances, the service provider network may need to scale up or down the number of VM instances. Further, the service provider network may need to route the requests to the appropriate VM instances, such as VM instances that are allocated for use by a particular user. The process of routing varying amounts of requests to the appropriate VM instances to serve the requests (e.g., execute a command) may also require scaling up or down the number of routers that are responsible for routing the requests to the VM instances. However, as the number of routers needed to route requests from users to appropriate VM instances scales up or down, various inefficiencies may be experienced in the service provider network.